Mobile communication devices, such as cellular phones and personal digital assistants (PDAs) with wireless communication capabilities, may be considered to be complex combinations of hardware, firmware and software. It is typical that such devices are developed according to established standards to promote use on networks provided by disparate service providers. It is also typical that mobile device firmware and software have many issues that need to be resolved while being developed. As such, development of mobile device firmware and software may be organized as a development cycle that includes an engineering build, a beta build and a commercial build. The act of debugging software often involves inserting debugging items in the code. Such debugging items often simply write some informative data to a log to indicate that execution of the code took a particular branch, perhaps additionally writing to the log a current value of one or more variables associated with the informative data to report the events/procedures of embedded processing and radio processing. Often debugging items are associated with condition judgments such that writing to the log is performed only when a certain condition is met. Typically, many debugging items are part of the engineering build of the operating system software for a mobile device. Later, fewer debugging items are built into the beta build. For reasons that will become clear hereinafter, the commercial build typically includes no debugging items. Decisions regarding how many debugging items to include in the code, and which debugging items to include in the code, are typically made at different times, while the operating system software is coded by different developers for various software modules, i.e., before the software is compiled. As such, only those debugging items included in the code after compiling and linking may write to the log. Clearly, further debugging items may not be added after the operating system software is compiled.
Even after a long development cycle, problems may still be encountered once a device is in use by an end user in a real-world situation. When such problems are reported, a mobile device manufacturer may send Inter-Operability Testing (IOT) specialists to the region of the end user that has reported the problem. The IOT specialists then attempt to duplicate the reported problem on an equivalent mobile device, which is connected via USB or serial bus to a log-collecting-software-loaded computer, running an engineering build of the operating system software, while the built-in debugging items record, in a log, various actions of the mobile device. The execution of the debugging items may be considered to assist “Runtime Troubleshooting”. The log may then be analyzed to learn the procedures that were executing at the time of the problem and other context information that may allow an IOT specialist to determine the source of the problem. Unfortunately, the extensive travel required by the sending of IOT specialists to duplicate every reported problem can be expensive. Additionally, it is not always possible to duplicate the reported problem, in part because the engineering build and the commercial build of the software may differ; and in part due to a fluctuating radio environment and different particular user operations with various procedure contexts.
Although it may be considered useful to have each device continuously generate debugging data, simply including all debugging items in the commercial build of the operating system software may lead to internal buffers overflowing with debugging data to which an IOT specialist may never refer. Without connection to a local computer for instant and dynamic debugging data flushing, such internal buffer overflow may happen in a time that may as short as to be measured in milliseconds and may cause the truly valuable logging data to be shifted out and overwritten by unrelated logging data. Due to the latency of multi-threading, the time gap between the triggering event and actual data log retrieval is often long enough that the original debugging data may be lost. Clearly, a system should be put in place to control the collection of the debugging data.